Exercise information display system, exercise information display method and computer-readable recording medium

ABSTRACT

Provided is an exercise information display system including a sensor unit attached to an ankle of one leg of a human body to output data on a motion state of the leg and a control unit processing the data. The control unit acquires a parameter based on data output from the sensor unit in a case where the human body performs a calibration motion for acquiring the parameter expressing at least one of an attachment orientation and an attachment position in the one leg of the sensor unit and a posture of the one leg in standing on the one leg, generates a reproduction image where a motion state of the leg during the exercise is reproduced in pseudo manner based on data output from the sensor unit in a case where the human body performs exercise of moving the leg and the parameter, and displays an image including at least the reproduction image as exercise information on the display unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present invention contains subject matter related to Japanese PatentApplication No. 2014-251931 filed in the Japanese Patent Office on Dec.12, 2014, the entire contents of which are incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an exercise information display systemand an exercise information display method capable of allowing a user tosimply and precisely determine an exercise state, and acomputer-readable recording medium storing an exercise informationdisplay program.

2. Description of the Related Art

In recent years, as health consciousness increases, an increasing numberof people do exercise such as running, walking, or cycling every day tomaintain or improve health state. In addition, an increasing number ofpeople also make a further full-scale training for the purpose ofparticipating in various games, competitions, or the like.

These people have much awareness and a high interest in measuring orrecording their own health state or exercise state as numerical valuesor data.

In order to meet these demands, in recent years, products or techniquesof determining a motion of a body by using a motion sensor capable ofmeasuring acceleration or angular velocity have been studied. Forexample, Japanese Patent Application Publication No. 2012-000343discloses a walking analysis system where a measurement sensor isinstalled to sandwich hip joints, knee joints, or ankle joints andacceleration or angular velocity during the walking is measured, so thatjoint angles of the joints or other walking information are determinedto perform estimation of the walking state.

In the technique disclosed in the above Document or the like, in orderto determine a motion of a lower limb during the exercise to reproducethe motion state, motion sensors of a hip, a thigh, and a lower leg areconfigured to be installed to measure acceleration, angular velocity, orthe like according to each motion. Therefore, a large number of themotion sensors need to be attached to the body, trouble thereof is verycomplicated, and construction of the system costs higher. Therefore,ordinary persons cannot easily use the system.

BRIEF SUMMARY OF THE INVENTION

The present invention has an advantage in that it is possible to providean exercise information display system, an exercise information displaymethod capable of precisely determining an exercise state, and acomputer-readable recording medium storing an exercise informationdisplay program, thus, accurately reproducing the exercise state of theuser by a simple configuration where a sensor unit is attached to onlyone ankle of a user.

According to an embodiment of the present invention, there is providedan exercise information display system including: a sensor unit which isattached to an ankle of one leg of a human body and outputs data on amotion state of the one leg; a display unit; and a control unit whichprocesses the data, wherein, the control unit performs: acquiring thedata as calibration data in a case where the human body performing acalibration motion for acquiring a parameter expressing at least one ofan attachment orientation of the sensor unit, an attachment position ofthe sensor unit on the one leg, and a posture of the one leg in standingon the one leg, acquiring the parameter based on the calibration data,acquiring the data as exercise data in a case where the human bodyperforming exercise of moving at least the one leg, generating areproduction image where a motion state of the leg during the exerciseis reproduced in a pseudo manner based on the exercise data and theparameter, and displaying an image including at least the reproductionimage as exercise information on the display unit.

According to another embodiment of the present invention, there isprovided an exercise information display method including the steps of:acquiring data output from a sensor unit as calibration data attached toan ankle of one leg of a human body in a case where the human bodyperforms a calibration motion for acquiring a parameter expressing atleast one of an attachment orientation of the sensor unit, an attachmentposition of the sensor unit on the one leg, and a posture of the one legin standing on the one leg; acquiring the parameter based on thecalibration data; acquiring the data as exercise data in a case wherethe human body performs exercise of moving at least the one leg;generating a reproduction image where a motion state of the leg duringthe exercise is reproduced in a pseudo manner based on the exercise dataand the parameter; and displaying an image including at least thereproduction image as exercise information on a display unit.

According to still another embodiment of the present invention, there isprovided a non-transitory computer-readable recording medium storing anexercise information display program for causing a computer to execute,wherein the exercise information display program causes the computer toexecute: acquiring data from a sensor unit as calibration data attachedto an ankle of one leg of the human body in a case where the human bodyperforms a calibration motion for acquiring a parameter expressing atleast one of an attachment orientation of the sensor unit, an attachmentposition of the sensor unit on the one leg, and a posture of the one legin standing one the one leg, acquiring the parameter based on thecalibration data; acquiring the data as exercise data in a case wherethe human body performs exercise of moving at least the one leg,generating a reproduction image where a motion state of the leg duringthe exercise is reproduced in a pseudo manner based on the exercise dataand the parameter; and displaying an image including at least thereproduction image as exercise information on a display unit.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

FIGS. 1A, 1B, and 1C are schematic configurational diagrams illustratingan embodiment of an exercise information display system according to thepresent invention;

FIGS. 2A and 2B are schematic block diagrams illustrating aconfigurational example of an sensor device and an informationprocessing device applied to the exercise information display systemaccording to one embodiment;

FIG. 3 is a flowchart illustrating an example of a control method(exercise information measuring operation) in the sensor device appliedto the exercise information display system according to one embodiment;

FIG. 4 is a flowchart illustrating an example of a control method(exercise information reproducing operation) in the informationprocessing device applied to the exercise information display systemaccording to one embodiment;

FIG. 5 is a flowchart illustrating an example of a calibration processapplied to the exercise information display system according to oneembodiment;

FIG. 6 is a flowchart illustrating an example of an attachmentorientation correction matrix estimation process for a sensor deviceexecuted in a calibration process applied to one embodiment;

FIG. 7 is a flowchart illustrating an example of a lower limb referenceposture setting process executed in the calibration process applied toone embodiment;

FIG. 8 is a flowchart illustrating an example of a lower limb lengthestimation process executed in the calibration process applied to oneembodiment;

FIG. 9 is a flowchart illustrating an example of a lower limb lengthacquisition method (lower limb length acquisition process) executed inthe lower limb length estimation process applied to one embodiment;

FIGS. 10A, 10B, and 10C are signal waveform diagrams illustrating anexample of an output signal of an angular velocity sensor acquiredduring bending and stretching motion executed in the attachmentorientation correction matrix estimation process applied to oneembodiment;

FIGS. 11A, 11B, and 11C are schematic diagrams illustrating an exampleof a bending motion executed in a lower limb length estimation processapplied to one embodiment;

FIG. 12 is a flowchart illustrating an example of a lower limb motionreproduction process applied to the exercise information display systemaccording to one embodiment;

FIG. 13 is a flowchart illustrating an example of a sensor deviceposture estimation process executed in the lower limb motionreproduction process according to one embodiment;

FIG. 14 is a flowchart illustrating an example of a sensor deviceposture estimation process executed in the lower limb motionreproduction process applied to one embodiment;

FIG. 15 is a flowchart illustrating an example of a lower limb motionestimation process executed in the lower limb motion reproductionprocess applied to one embodiment;

FIGS. 16A, 16B, 16C, 16D, and 16E are signal waveform diagrams forexplaining a cutting-out process for one cycle of running motion in thelower limb motion reproduction process according to one embodiment;

FIGS. 17A and 17B are schematic diagrams illustrating a posture of auser during the cutting-out of one cycle of running motion in the lowerlimb motion reproduction process applied to one embodiment;

FIG. 18 is a schematic diagram illustrating a position of a referenceposture of one cycle of running motion in the lower limb motionreproduction process applied to one embodiment;

FIG. 19 is a schematic diagram illustrating an example of a table of arelationship of correspondence between a lower leg length and a motionof a base of a thigh referred to in the lower limb motion reproductionprocess applied to one embodiment;

FIG. 20 is a stick model illustrating one cycle of motion of two lowerlimbs estimated in the lower limb motion reproduction process applied toone embodiment;

FIG. 21 is a schematic diagram illustrating an example of exerciseinformation displayed in an information processing device applied to theexercise information display system according to one embodiment; and

FIGS. 22A, 22B, and 22C are schematic configurational diagramsillustrating a modified example of the exercise information displaysystem according to one embodiment.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, embodiments of an exercise information display system andan exercise information display method according to the presentinvention will be described in detail with reference to the drawings.

In addition, in the below description, the case where a user does anexercise such as running will be described.

<Exercise Information Display System>

FIGS. 1A, 1B, and 1C are schematic configurational diagrams illustratingan embodiment of an exercise information display system according to thepresent invention. FIG. 1A is a schematic diagram illustrating anexample of attachment of a sensor device on a human body, which isapplied to the exercise information display system. FIG. 1B is aschematic diagram illustrating a whole configuration of the exerciseinformation display system. FIG. 1C is a schematic diagram illustratinga coordinate system (sensor coordinate system) in a sensor device.

FIGS. 2A and 2B are schematic block diagrams illustrating aconfigurational example of a sensor device and an information processingdevice applied to the exercise information display system according tothe embodiment.

For example, as illustrated in FIGS. 1A and 1B, the exercise informationdisplay system according to the embodiment is configured to include asensor device 100 which the user US as a measurement object attached toa body and an information processing device 200 which reproduces in apseudo manner an exercise state based on sensor data and the likeacquired from the sensor device 100 and visibly provides the exercisestate to the user.

(Sensor Device 100)

As illustrated in FIG. 1A, the sensor device 100 is attached to an ankleof one leg of the user US or in the vicinity of the ankle.

For example, as illustrated in FIG. 1B, the sensor device 100 has anouter appearance mainly including a device main body 101 includingvarious sensors detecting an exercise state of the user US and a beltunit 102 winding around the ankle of one leg of the user US to attachthe device main body 101 on the ankle.

In addition, in FIGS. 1A and 1B, as a configuration of the sensor device100, the case where the device main body 101 is attached by winding thebelt unit 102 on the ankle of one leg of the user US is described.

However, the invention is not limited thereto. Namely, with respect tothe sensor device 100, if a sensor device has a structure to be attachedto a predetermined site (herein, in the vicinity of the ankle) of thebody of the user US without occurrence of irregularity, the sensordevice may have a different configuration.

For example, the sensor device may not include the above-described beltunit 102, and the sensor device may have a configuration where thesensor device is attached by adhering to shoes or clothing (wear, socks,or the like) which is worn by the user US, being embedded inside, orbeing sandwiched by a clip or the like.

More specifically, for example, as illustrated in FIG. 2A, the sensordevice 100 is mainly configured to include a sensor unit 110, an inputmanipulation unit 120, a notification unit 130, a control unit(calibration process unit) 140, a storage unit 150, a communicationinterface unit (hereinafter, referred to as a “communication I/F unit”)160, and a power supply unit 170.

The sensor unit 110 includes motion sensors for detecting a motion of ahuman body. As illustrated in FIG. 2A, the sensor unit includes at leastan acceleration sensor 112 and an angular velocity sensor (gyro sensor)114.

The acceleration sensor 112 measures a rate of change in motion velocity(acceleration of a translation motion) during the exercise of the userUS and outputs acceleration data (acceleration signals) in threeperpendicular axis directions

The angular velocity sensor 114 measures a rate of change in motionorientation (rotational angular velocity) during the exercise of theuser US and outputs angular velocity data (angular velocity signals) inthree perpendicular axis directions.

The acceleration sensor 112 and the angular velocity sensor 114 performmeasurement every set sampling interval. The sampling interval is setto, for example, 5 msec or about thereof.

Sensor data (acceleration data, angular velocity data) are measured andoutput by the sensor unit 110 at least when the user US does exercise ofmoving one leg attached with the sensor device 100, and the sensor dataare stored in a predetermined storage area of the later-describedstorage unit 150.

Herein, a sensor coordinate SNC of the acceleration sensor 112 and theangular velocity sensor 114 is a relative coordinate, and for example,as illustrated in FIG. 1C, on the basis of an attachment position of thesensor device 100, a direction toward the knee (upward direction in thefigure) is set as the +Z axis direction, an advancement direction in theerect state (leftward direction in the figure) is set as the +X axisdirection, and a left-handed direction in the erect state (frontwarddirection in the figure) is set as the +Y axis direction.

In contrast, a world coordinate WDC is an absolute coordinate, and asillustrated in FIG. 1A, an advancement direction during the pacing(leftward direction in the figure) is set as the +Z axis direction, aleft-handed direction during the pacing (frontward direction in thefigure) is set as the +X axis direction, and a direction vertical to thegroup (upward direction in the figure) is set as the +Y axis direction.Herein, the −Y axis direction in the world coordinate WDC corresponds tothe gravity direction (direction of the gravitational acceleration).

For example, as illustrated in FIG. 1B, the input manipulation unit 120includes manipulation buttons (push buttons) 122 and 124 and a switchsuch as a slide button or a touch button which are installed on a sidesurface, a front surface, or the like of the device main body 101.

The input manipulation unit 120 is used, for example, for powermanipulation for activating the sensor device 100 and manipulation forsetting an operation mode of the sensor device 100.

Although not shown, the notification unit 130 includes a sound unit suchas a buzzer or a speaker, a vibration unit such as a vibration motor ora vibrator, a light emitting unit of a light emitting device or the likesuch as an LED, which is installed in the device main body 101.

The notification unit 130 provides and notifies at least thelater-described information on procedure in the calibration process forpredetermined parameters or information on operation abnormality of thesensor device 100 to the user through visual perception, auditoryperception, tactile perception, and the like.

In addition, the notification unit 130 may include at least one of thesound unit, the vibration unit, the light emitting unit, and the like ora combination of the plural units.

The control unit 140 is a processing unit such as a CPU (centralprocessing unit) or an MPU (microprocessor) having a timing function andexecutes a predetermined control program or algorithm program on thebasis of an operation clock.

Thus, the control unit 140 controls various operations such as acalibration process for a predetermined parameter, a sensing operationof the sensor device 100, a notification operation of the notificationunit 130, and a transmission operation for the sensor data or the likein the communication I/F unit 160.

Herein, in the embodiment, when the user US performs predetermined inputmanipulation by using the input manipulation unit 120, the control unit140 sets a predetermined operation mode and executes an operationaccording to the operation mode.

In addition, the operation in each operation mode will be describedlater.

The storage unit 150 stores the sensor data or the like acquired fromthe above-described sensor unit 110 in association with time data in apredetermined storage area.

When the control unit 140 executes a predetermined program, the storageunit 150 stores various data or information used or generated in thecalibration process for a predetermined parameter, the sensing operationof the sensor unit 110, the transmission operation for the sensor datain the communication I/F unit 160, or the like.

The storage unit 150 further stores the control program or the algorithmprogram executed in the control unit 140.

In addition, the program executed in the control unit 140 may beincorporated in the control unit 140 in advance.

A portion or entire portions of the storage unit 150 may have a form ofa removable storage medium such as a memory card or may be configured tobe detachable to the sensor device 100.

The communication I/F unit 160 functions as an interface at the time oftransmitting/receiving the sensor data (acceleration data, angularvelocity data) acquired from the sensor unit 110, various data orinformation generated in the later-described calibration process, or thelike to/from the information processing device 200.

Herein, as a method of transmitting/receiving the sensor data or thelike between the sensor device 100 and the information processing device200 through the communication I/F unit 160, for example, variouswireless communication methods or wired communication methods may beapplied.

Herein, in the communication I/F unit 160, in the case oftransmitting/receiving the sensor data or the like in the wirelesscommunication method, for example, Bluetooth (registered trademark) as alocal area wireless communication standard for a digital device,Bluetooth (registered trademark) low energy (LE) set as a low powerconsumption type communication, NFC (Near field communication), orequivalent communication methods can be applied well.

The power supply unit 170 supplies driving power to each component ofthe sensor device 100. As the power supply unit 170, for example, acommercially available primary battery such as a button cell or acommercially available secondary battery such as a lithium ion batteryis applied.

Besides the above-described primary battery or secondary battery, as thepower supply unit 170, power sources according to an energy harvestingtechnique for generating electricity by energy of vibration, light,heat, electromagnetic waves, or the like alone or in combination withother power sources.

(Information Processing Device 200)

After the end of the exercise of the user US, the information processingdevice 200 reproduces in a pseudo manner the exercise state(particularly, the motion state of the lower limb) of the body based onthe sensor data or the like transmitted from the sensor device 100 andprovides the reproduced exercise state to the user US in a predetermineddisplay form.

Herein, the information processing device 200 includes at least adisplay unit and has a function of capable of executing a predeterminedprogram (exercise information display program) for reproducing in apseudo manner the motion state of the lower limb described later. Asillustrated in FIG. 1B, the information processing device 200 may be anotebook type personal computer or desktop type personal computer or maybe a mobile information terminal such as a smartphone (highly-functionalmobile phone) or a tablet terminal.

In the case where the information processing device 200 executes anexercise information display program for reproducing the motion state ofthe lower limb by using a server computer (so called, a cloud system) ona network, the information processing device 200 may have a function ofconnecting to the network and may be a communication terminal having anetwork information browsing software.

More specifically, for example, as illustrated in FIG. 2B, theinformation processing device 200 is mainly configured to include adisplay unit 210, an input manipulation unit 220, a control unit (motionreproduction process unit) 240, a storage unit 250, a communication I/Funit 260, and a power supply unit 270.

The display unit 210 includes, for example, a liquid crystal type orlight-emitting type display panel and displays at least informationassociated with the input manipulation using the input manipulation unit220 or the exercise state (motion state of the lower limb) reproduced ina pseudo manner based on the sensor data or the like in a predeterminedform.

The input manipulation unit 220 is an input unit such as a keyboard, amouse, a touch pad, or a touch panel attached to the informationprocessing device 200. The input manipulation unit 220 is used at thetime of executing a function corresponding to an icon, a menu, or aposition by selecting an arbitrary icon or menu displayed on the displayunit 210 or indicating an arbitrary position in a displayed screen.

The control unit 240 is a processing unit such as a CPU or an MPU andexecutes a predetermined control program or a predetermined algorithmprogram to control a process of reproducing the motion state of thelower limb during the exercise of the user US and various operations ofthe display operation of the display unit 210 and the receptionoperation of the sensor data or the like in the communication I/F unit260.

The storage unit 250 stores the sensor data or the like transmitted fromthe sensor device 100 through the communication I/F unit 260 in apredetermined storage area. When the control unit 240 executes apredetermined program, the storage unit 250 stores various data orinformation used or generated in the process of reproducing the motionstate of the lower limb during the exercise of the user US, the displayoperation of the display unit 210, the reception operation of the sensordata or the like in the communication I/F unit 260.

The storage unit 250 further stores the control program or the algorithmprogram executed in the control unit 240.

In addition, the program executed in the control unit 140 may be builtin the control unit 140 in advance.

A portion or entire portions of the storage unit 250 may have a form ofa removable storage medium such as a memory card or may be configured tobe detachable to the information processing device 200.

The communication I/F unit 260 functions as an interface at the time oftransmitting/receiving the sensor date acquired from the sensor device100, various data or information generated in the calibration process,or the like. Herein, as a method of transmitting/receiving the sensordata or the like between the information processing device 200 and thesensor device 100 through the communication I/F unit 260, as describedabove, various wireless communication methods or wired communicationmethods may be applied.

The power supply unit 270 supplies driving power to each component ofthe information processing device 200. As the power supply unit 270, asecondary battery such as a lithium ion battery or a commercial AC powersource is applied for a mobile electronic device such as a smartphone, atablet terminal, or a notebook type personal computer. In the case wherethe information processing device 200 is a desktop type personalcomputer, as the power supply unit 270, a commercial AC power source isapplied.

<Exercise Information Display Method>

Next, the control method (exercise information display method) in theexercise information display system according to the embodiment will bedescribed with reference to the drawings.

FIG. 3 is a flowchart illustrating an example of a control method(exercise information measuring operation) in the sensor device appliedto the exercise information display system according to the embodiment.

FIG. 4 is a flowchart illustrating an example of a control method(exercise information reproducing operation) in the informationprocessing device applied to the exercise information display systemaccording to the embodiment.

In the exercise information display system according to the embodiment,the exercise information measuring operation of the sensor device 100illustrated in the flowchart of FIG. 3 and the exercise informationreproducing operation of the information processing device 200illustrated in the flowchart of FIG. 4 are mainly executed.

Herein, a series of the control methods in the following operations arerealized by the sensor device 100 and the information processing device200 executing predetermined programs.

In the exercise information measuring operation of the sensor device100, as illustrated in FIG. 3, first, the user US attaches the sensordevice 100 to the body and activates the sensor device 100 (step S102).

More specifically, the user US attaches the device main body 101 to apredetermined position, for example, by winding the belt unit 102 of thesensor device 100 on the left ankle.

Next, the user US performs power-on manipulation on a power switch(input manipulation unit 120) of the device main body 101, so thatdriving power is supplied from the power supply unit 170 to eachcomponent of the sensor device 100, and thus, the sensor device 100 isactivated.

In addition, in the sensor device 100, a battery power source (primarybattery or charged secondary battery) may be attached to the powersupply unit 170 supply the driving power from the power supply unit 170to each component, so that the sensor device 100 may be set in aconstantly activated state.

In this case, when the state that the sensing operation, the inputmanipulation, or the like is not performed continues for a predeterminedtime after the driving power is supplied from the power supply unit 170,it is preferable that the sensor device 100 proceed to a sleep state orthe like so as to reduce power consumption.

Next, the user US manipulates the manipulation buttons 122 and 124 ofthe input manipulation unit 120 installed to the device main body 101(step S104), so that the sensor device 100 is set to a predeterminedoperation state (operation mode).

Next, the control unit 140 determines the manipulation state of theinput manipulation unit 120 by the user US (step S106) to set theoperation mode according to the manipulation state.

More specifically, in the case where the control unit 140 determinesthat, for example, the manipulation button 122 of the input manipulationunit 120 is pushed by the user US (input manipulation A) (step S108),the control unit controls the sensor device 100 to proceed to thecalibration mode, and the calibration process for a predeterminedparameter is executed (step S110).

The control unit 140 controls various data or information generated bythe calibration process to be stored in the predetermined storage areaof the storage unit 150. In addition, the calibration process will bedescribed later.

In the case where the control unit 140 determines that, for example, themanipulation button 124 is pushed by the user US (input manipulation B)(step S112), and the control unit controls the sensor device 100 toproceed to the measurement mode.

Next, the control unit controls the sensor unit 110 to start the sensingoperation to acquire the sensor data (acceleration data, angularvelocity data) during the exercise (practicing) of the user US andcontrols the acquired sensor data in association with time data to bestored in the predetermined storage area of the storage unit 150 (stepS114).

In addition, in the case where the manipulation button 124 is pushed bythe user US again during the measurement mode, the control unit 140 endsthe measurement mode.

In the case where the control unit 140 determines that, for example, themanipulation button 122 is long-pushed for 5 seconds or more by the userUS (input manipulation C) (step S116), the control unit controls thesensor device 100 to proceed to the communication mode.

Next, the control unit controls the sensor data during the exercisestored in the storage unit 150 or various data or information generatedby the calibration process to be transmitted to the informationprocessing device 200 according to a predetermined communication method(step S118).

In addition, the communication mode is generally set after the end ofthe exercise by the user US, and the sensor data or the like stored inthe storage unit 150 are transmitted to the information processingdevice 200. For this reason, when the sensor device 100 is in thiscommunication mode, the sensor device may not be attached to the body(ankle) of the user US.

In the case where the control unit 140 determines that, for example, themanipulation buttons 122 and 124 are simultaneously long-pushed by theuser US (input manipulation D) (step S120), the control unit controlsthe sensor device 100 to proceed to the sleep mode so that the sensordevice 100 is operated in a low power consumption state (step S122).

Next, as illustrated in the flowchart of FIG. 3, after the calibrationprocess in the calibration mode (step S110), the sensor data acquisitionoperation in the measurement mode (step S114), and the transmissionoperation for the sensor data or the like in the communication mode(step S118) among the above-described operation modes, the control unit140 returns to step S104 to wait for the next input manipulation of theuser US (stand by).

At this time, after the end of the above-described process operations(the calibration process, the sensor data acquisition operation, and thetransmission operation for the sensor data or the like), the controlunit 140 may allow the sensor device 100 to proceed to the sleep mode toset the sensor device 100 to the state where the sensor device isoperated in the low power consumption state so as to wait for the nextinput manipulation of the user US.

In the case where the low power consumption operation in the sleep mode(step S122) among the above-described operation modes continues for apredetermined time, the control unit 140 ends the exercise informationmeasuring operation of the sensor device 100.

At this time, after the sensor device 100 is allowed to proceed to thesleep state, the control unit 140 may turn to step S104 so as to waitfor the next input manipulation of the user US.

In addition, although not shown in the flowchart illustrated in FIG. 3,the control unit 140 constantly monitors input manipulation of stoppingor ending a process operation or a change of the operation state duringthe execution of a series of the process operations in the sensor device100, and in the case where the input manipulation or the change of thestate is detected, the control unit forcibly ends the process operation.

More specifically, when the control unit 140 detects power-offmanipulation of the power switch by the user US, a decrease in remainingamount of battery in the power supply unit 170, or abnormality offunction or application during the execution of the process operation,the control unit forcibly stops and ends a series of the processoperations.

As illustrated in FIG. 4, with respect to an exercise informationreproducing operation in the information processing device 200, first,the user US performs power-on manipulation on the power switch (inputmanipulation unit 220) of the information processing device 200, so thatdriving power is supplied from the power supply unit 270 to eachcomponent of the information processing device 200, and thus, theinformation processing device 200 is activated (step S202).

Next, by the user US manipulating the input manipulation unit 120 of thesensor device 100, the control unit 240 receives the sensor data or thelike transmitted from the sensor device 100 set to the communicationmode according to a predetermined communication method through thecommunication OF unit 260 (step S204).

Herein, the sensor data acquired during the exercise or various data orinformation generated by the calibration process which are transmittedfrom the sensor device 100 are stored in the predetermined storage areaof the storage unit 250.

Next, the control unit 240 executes a lower limb motion reproductionprocess of reproducing in a pseudo manner the motion state of the lowerlimb during the exercise of the user US based on the sensor data(acceleration data and angular velocity data) and various data orinformation generated by the calibration process which are transmittedfrom the sensor device 100 (step S206).

In addition, the process of reproducing in a pseudo manner the motionstate of the lower limb of the user US will be described later.

Next, the control unit 240 controls the currently reproduced motionstate of the lower limb of the user US, the motion state of the lowerlimb during the previous exercise of the user US stored in advance inthe storage unit 250, or the motion states of lower limbs of athleteshaving a form over an arbitrary level to be displayed on the displayunit 210 in a comparable form (step S208).

Thus, by viewing the comparison animation displayed on the display unit210, the user US can precisely determine the motion state during theexercise and can reflect on the subsequent training or the like.

(Calibration Process)

Next, the calibration process executed in the control method (exerciseinformation measuring operation) of the above-described exerciseinformation display system will be described in detail with reference tothe drawings.

FIG. 5 is a flowchart illustrating an example of a calibration processapplied to the exercise information display system according to theembodiment.

FIG. 6 is a flowchart illustrating an example of an attachmentorientation correction matrix estimation process for the sensor deviceexecuted in the calibration process applied to the embodiment.

FIG. 7 is a flowchart illustrating an example of a lower limb referenceposture setting process executed in the calibration process applied tothe embodiment.

FIG. 8 is a flowchart illustrating an example of a lower limb lengthestimation process executed in the calibration process applied to theembodiment.

FIG. 9 is a flowchart illustrating an example of a lower limb lengthacquisition method (lower limb length acquisition process) executed inthe lower limb length estimation process applied to the embodiment.

FIGS. 10A, 10B, and 10C are signal waveform diagrams illustrating anexample of an output signal of an angular velocity sensor acquiredduring bending/stretching motion executed in the attachment orientationcorrection matrix estimation process applied to the embodiment.

FIGS. 11A, 11B, and 11C are schematic diagrams illustrating an exampleof bending motion executed in the lower limb length estimation processapplied to the embodiment.

In the calibration process applied to the exercise information displaysystem according to the embodiment, as illustrated in the flowchart ofFIG. 5, a process regarding application output of the sensor device 100(attachment orientation correction matrix estimation process; stepS302), a process regarding reference posture of a lower limb (lower limbreference posture setting process; step S304) and a process regarding alength of a lower limb (lower limb length estimation process; step S306)are mainly executed.

Herein, a series of the following process operations are realized byexecuting a predetermined program in the sensor device 100.

In the attachment orientation correction matrix estimation process forthe sensor device 100 according to the embodiment, as illustrated inFIG. 6, first, the control unit 140 stands by, for example, for 1 secondin the state where the control unit starts a sensing operation in theacceleration sensor 112 and the angular velocity sensor 114 of thesensor unit 110 (step S322), and after that, the control unit controlsthe notification unit 130 to generate a first buzzer sound having alength of, for example, about 0.5 seconds (step S324).

Next, for example, after 1 second, the control unit 140 controls thenotification unit 130 to generate a second buzzer sound having a lengthof, for example, about 1 second (step S328).

Herein, during the period (for example, 1 second) from the time when thefirst buzzer sound is generated from the notification unit 130 to thetime when the second buzzer sound is generated, the user US maintain astate where the user is erect and unmoving (step S326).

Next, after the second buzzer sound, the user US performs abending/stretching motion in a state that the user keeps two kneestogether (in close contact with each other) from the erect state and,after that, performs a motion of returning to the erect state again(step S330).

The control unit 140 executes a correction process for a difference ofthe attachment orientation of the sensor device 100 to the ankle fromcorrect attachment orientation based on a series of the motions of theuser US.

Herein, in the embodiment, the state where the X axis of the sensorcoordinate SNC (refer to FIG. 1C) of the sensor device 100 attached tothe ankle of the user US is coincident with an advancement directionduring the pacing of the user US who is erect, that is, the Z axis ofthe world coordinate WDC (refer to FIG. 1A) is defined as the correctattachment orientation of the sensor device 100.

On the basis of this, when a real attachment orientation of the sensordevice 100 is shifted from the above-described correct attachmentorientation, the control unit 140 obtains a matrix (attachmentorientation correction matrix) Rc for converting the sensor data outputfrom the sensor unit 110 to sensor data of the case where the sensordevice 100 is to be attached with the correct attachment orientation.

Herein, in the state where the attachment orientation of the sensordevice 100 is shifted from the correct attachment orientation, when theuser US performs the bending/stretching motion in the above-describedstep S330, signal waveforms output from the angular velocity sensor 114are illustrated in, for example, FIGS. 10A, 10B, and 10C.

FIG. 10A illustrates a temporal change of an angular velocity signal ofthe X axis, FIG. 10B illustrates a temporal change of an angularvelocity signal of the Y axis, and FIG. 10C illustrates a temporalchange of an angular velocity signal of the Z axis.

In FIGS. 10A and 10B, among the characteristic waveform changes Wx andWy observed in the angular velocity signal waveforms of the X and Yaxes, the Wx is caused from that fact that the sensor device 100 is notat the correct attachment orientation with respect to the ankle and thereal attachment orientation is shifted in the rotational direction ofthe Z axis from the correct attachment orientation.

In addition, in the case where the sensor device 100 is attached at thecorrect orientation with respect to the ankle, the angular velocitysignal of the X axis output from the angular velocity sensor 114 is in asubstantially constant (flat) state, and the characteristic waveformchange Wy is observed from only the angular velocity of the Y axis.

Thus, in the embodiment, as illustrated in FIGS. 10A, 10B, and 10C, theattachment orientation correction matrix Rc for the sensor device 100 isobtained by using a half-period section Ic of the characteristicwaveform change Wx observed during the bending/stretching motion of theuser US.

First, the control unit 140 estimates a rotation axis of thebending/stretching motion (step S332).

Herein, an average value of the half-period section Ic of thecharacteristic waveform change Wx observed from the signal waveformsoutput from the angular velocity sensor 114 is normalized as aveGyr1.

Herein, as the aveGyr1 is viewed as a vector, the aveGyr1 becomes therotation axis of the bending/stretching motion in the sensor coordinateSNC.

Thus, the control unit 140 obtains the convert matrix (attachmentorientation correction matrix) Rc for converting the rotation axisaveGyr1 to the rotation axis vecY [0 1 0] of the case where the sensordevice 100 is attached to the correct position by using a series of thefollowing group of equations (step S334).vec2=cross(aveGyr1,vecY)rad1=a cos(dot(aveGyr1,vecY))rvec1=vec2×rad1

Herein, the rvec1 is the rotation vector, and conversion from therotation vector (axis-angle) to the rotation matrix is the attachmentorientation correction matrix Rc.

Herein, cross denotes outer product calculation, dot denotes innerproduct calculation, and a cos denotes inverse cosine calculation.

The conversion from the rotation vector to the rotation matrix isperformed through a quaternion q as follows.

Herein, the quaternion q is a four-element number and is represented byq=[qw, qx, qy, qz].

As expressed in the above equations, since the rotation vector rvec1 isconfigured with the rotation axis vec2 and the rotation angle rad1, thefollowing relationship is satisfied.qw=cos(0.5×rad1)qvec=sin(0.5×rad1)/norm(vec2)

Herein, qvec=[qx qy qz], and norm denotes a Euclid norm (a concept of alength) of a vector.

Thus, the attachment orientation correction matrix Rc can be expressedby the following equation (11).

Herein, the attachment orientation correction matrix Rc corresponds to adifference between the rotation axis of the case where the sensor device100 is attached to be shifted from the correct attachment orientationand the rotation axis of the case where the sensor device 100 isattached with the correct attachment orientation.

$\begin{matrix}{\mspace{76mu}\left\lbrack {{Mathematical}\mspace{14mu}{Formula}\mspace{14mu} 1} \right\rbrack} & \; \\{{Rc} = {\begin{bmatrix}1 & 0 & 0 \\0 & 1 & 0 \\0 & 0 & 1\end{bmatrix} + {2 \times \begin{bmatrix}{{- {qy}^{2}} - {qz}^{2}} & {{{qx} \times {qy}} - {{qw} \times {qz}}} & {{{qx} \times {qz}} + {{qw} \times {qy}}} \\{{{qx} \times {qy}} + {{qw} \times {qz}}} & {{- {qx}^{2}} - {qz}^{2}} & {{{qy} \times {qz}} - {{qw} \times {qx}}} \\{{{qx} \times {qz}} - {{qw} \times {qy}}} & {{{qy} \times {qz}} + {{qw} \times {qx}}} & {{- {qx}^{2}} - {qy}^{2}}\end{bmatrix}}}} & (11)\end{matrix}$

Next, the lower limb reference posture setting process for the user USwill be described with reference to the flowchart illustrated in FIG. 7.

In the embodiment, the reference posture denotes a posture used areference at the time of recovering the motion state of the lower limbof the user US in the lower limb motion reproduction process executed inthe information processing device 200 described later.

In the embodiment, a posture of a left leg at the time of one-legstanding with the left leg attached with the sensor device 100 describedlater is used as the reference posture.

The control unit 140 executes a process of removing a gravitationalacceleration component from the acceleration data acquired from theacceleration sensor 112 or a process of setting an absolute posture atthe time of integrating the angular velocity data acquired from theangular velocity sensor 114 by using the reference posture.

In addition, a specific method of using the reference posture will bedescribed later in the lower limb motion reproduction process.

In the lower limb reference posture setting process, as illustrated inFIG. 7, first, the control unit 140 stands by, for example, for 1 secondin the state where the control unit starts the sensing operation in theacceleration sensor 112 and the angular velocity sensor 114 of thesensor unit 110 (step S342), and after that, the control unit controlsthe notification unit 130 to generate the first buzzer sound (stepS344).

Next, for example, after several seconds, the control unit 140 controlsthe notification unit 130 to generate the second buzzer sound (stepS348).

During the period between the first buzzer sound and the second buzzersound, the user US performs one-leg stand of taking off the right legfrom the ground in the state where the sole of the foot of the left legattached with the sensor device 100 is in contact with the ground (stepS346).

At this time, it is preferable that the lower leg from the knee to theankle of the left leg be allowed to be as erect as possible. The postureof the left leg at the time of performing the one-leg stand is used asthe reference posture.

In the above-described one-leg stand, if the lower leg is in the erectstate as viewed from the horizontal direction, that is, the X axisdirection of the world coordinate WDC, the acceleration componentsacquired from the acceleration sensor 112 appear only in the Y axis andthe Z axis, but the acceleration component does not appear in the X axiscorresponding to the advancement direction.

Thus, in the embodiment, the rotation element of the X axis in thesensor coordinate SNC is extracted from the reference posture.Therefore, the control unit 140 obtains a lower limb reference posturematrix R5 generated from the rotation element of the X axis (step S350).

A rotation angle angle 5 of the X axis in the reference posture isobtained by the following equation.angle5=−a sin(accy/accz)

Herein, a sin denotes inverse sine calculation, accy denotes anacceleration component of the Y axis at the time of performing one-legstand, accz denotes an acceleration component of the Z axis.

These components are the acceleration components of the Y and Z axes ofthe corrected acceleration data acc obtained by converting theacceleration data acc_in acquired from the acceleration sensor 112 byusing the above-described attachment orientation correction matrix Rc asfollows.acc=Rc×acc_in

Thus, the lower limb reference posture matrix R5 can be expressed by thefollowing equation (12).

$\begin{matrix}\left\lbrack {{Mathematical}\mspace{14mu}{Formula}\mspace{14mu} 2} \right\rbrack & \; \\{{R\; 5} = \begin{bmatrix}1 & 0 & 0 \\0 & {\cos\left( {{angle}\; 5} \right)} & {- {\sin\left( {{angle}\; 5} \right)}} \\0 & {\sin\left( {{angle}\; 5} \right)} & {\cos\left( {{angle}\; 5} \right)}\end{bmatrix}} & (12)\end{matrix}$

In addition, the case of simply one-leg standing with the left legattached with the sensor device 100 in the above-described lower limbreference posture setting process is described. However, the embodimentis not limited to this method, but the user may perform a motion ofjumping, for example, forward or upward, landing with only the left legattached with the sensor device 100, and stopping at the same time ofthe landing.

Next, the lower limb length estimation process for the user US will bedescribed with reference to flowcharts illustrated in FIGS. 8 and 9.

In the embodiment, the lower leg length denotes a length from aninstallation position of the acceleration sensor 112 installed in thesensor device 100 attached in the vicinity of the ankle of the user USto the knee.

In the lower limb length estimation process, as illustrated in FIG. 8,first, the control unit 140 stands by, for example, for 1 second in thestate where the control unit starts the sensing operation of theacceleration sensor 112 and the angular velocity sensor 114 in thesensor unit 110 (step S362), and after that, the control unit controlsthe notification unit 130 to generate the first buzzer sound having alength of, for example, about 0.5 seconds (step S364).

Next, for example, after several seconds, the control unit 140 controlsthe notification unit 130 to generate the second buzzer sound having alength of, for example, about 1 second (step S368).

During the period between the first buzzer sound and the second buzzersound, the user US moves (bends the knee) from the state illustrated inFIG. 11A where the lower leg DLr of the left leg attached with thesensor device 100 is erect to the position illustrated FIG. 11B wherethe position of the knee JTc is not moved as much as possible and thelower leg DLr is rotated 90° backward (the opposite direction of theadvancement direction; rightward in the figure) by using the knee JTc asthe rotation axis.

After that, as illustrated in FIG. 11A, a series of bending motions ofreturning the lower leg DLr to the position where the left leg isinitially erect are performed (step S366).

In the state illustrated in FIG. 11A, it is preferable that the lowerleg DLr is erect to the utmost.

In the bending motion, in order to bend the lower leg DLr by using theknee JTc as an axis, it is preferable that the position of the knee JTcis not moved to the utmost.

In addition, in FIGS. 11A, 11B, and 11C, JTa denotes a chest of thebody, JTb denotes a base of the thigh (leg), and JTd denotes an ankleattached with the sensor device 100.

Next, the control unit 140 performs a process of acquiring the lower leglength based on the sensor data acquired at the time when the user USperforms a series of the bending motions (period between the firstbuzzer sound and the second buzzer sound) as follows (step S370).

In the process of acquiring the lower limb length, as illustrated inFIG. 9, first, the control unit 140 the posture matrix R7 for convertingthe sensor coordinate SNC which is a relative coordinate to the worldcoordinate WDC1 which is equal to the sensor coordinate SNC in the erectstate based on the angular velocity data (step S382).

The world coordinate WDC1 becomes an absolute coordinate used only atthe time of acquiring the lower limb length. The X axis of the worldcoordinate WDC1 is the advancement direction, the Y axis is aleft-handed direction of the advancement direction, and the Z axis is aceiling direction perpendicular to the X axis and the Y axis.

More specifically, the control unit 140 converts the angular velocitydata to a rotation vector rvec6 as follows.angle6=norm(gyr)×dtvec6=gyr/norm(gyr)

Herein, the angle 6 is a rotation angle about the rotation axis vec6.

The dt denotes a sampling period (sampling time) of the angular velocitysensor 114 and is set to, for example, 5 msec.

The gyr is a corrected angular velocity data obtained by converting theangular velocity data gyr_in acquired from the angular velocity sensor114 by using the above-described attachment orientation correctionmatrix Rc as expressed in the following equation.gyr=Rc×gyr_in

The rotation vector rvec6 is expressed by the following equation byusing the rotation axis vec6 and the rotation angle angle6 describedabove.rvec6=vec6×angle6

Thus, conversion of the rotation vector rvec6 to the rotation matrix byusing the method of converting the rotation vector illustrated in theabove-described attachment orientation correction matrix (Rc) estimationprocess to the rotation matrix is the R6.

The rotation matrix used herein is a rotation matrix per time of thesampling period of the angular velocity sensor 114, and since therotation matrix exists in every sampling motion, the rotation matrix isexpressed by R6 {n}. Herein, n denotes a position (that is, frame) ofthe sampling motion.

Next, by integrating the above-described rotation matrix R6 {n}, thecontrol unit 140 obtains a posture matrix R7 {n} of the position where achange in the reference state appears.

Herein, when the front (initial) posture is used as a reference, theposture matrix R7{n} is calculated by integrating the rotation matrix R6{n} over all the sampling motions.

For example, the posture matrix R7{3} at the time of the third samplingmotion is expressed by the following equation.R7{3}=R7{2}×R6{3}=R6{1}×R6{2}×R6{3}

Next, the control unit 140 obtains the value (coordinate-convertedacceleration data) acc_w1 of the world coordinate WDC1 by performingcoordinate conversion on the acceleration data acc_in acquired from theacceleration sensor 112 by using the above-described posture matrix R7as expressed by the following equation (step S384).acc_w1=R7×acc_in

Herein, in the acceleration data acc_in acquired from the accelerationsensor 112 during the real pacing motion, the sensor coordinate SNC isalways changed with respect to the world coordinate WDC1 due to the legmotion associated with the pacing motion. Therefore, the direction ofthe gravitational acceleration is also changed.

On the contrary, as described above, in the coordinate-convertedacceleration data acc_w1 obtained by conversion to the world coordinateWDC1 by using the posture matrix R7 , the gravitational acceleration isalways exerted in the Z axis direction.

Next, the control unit 140 obtains the acceleration data acc configuredwith only the motion component associated with the motion (herein, aseries of the bending motions described above) by removing thegravitational acceleration component from the coordinate-convertedacceleration data acc_w1 as expressed by the following equation (stepS386).

Herein, the acceleration data acc are values obtained by using the worldcoordinate WDC1 as a reference.acc=acc_w1−[0 0 9.8]

Herein, the unit of the acceleration is m/s².

Next, the control unit 140 converts the acceleration data acc to aposition pos through two-times integration of the acceleration data accof only the motion component (step S388) to obtain the position of theankle at the time of each sampling motion.

Herein, if a condition that first and final positions in a series of thebending motions by the user US are equal to each other is applied, thelower leg length is calculated as expressed below.

Namely, first, the control unit 140 calculates a velocity vel throughone-time integration of the acceleration data acc configured with onlythe motion component and calculates the position pos through anotherintegration.vel=Σ((acc−offset)×dt)pos=Σ(vel×dt)

Herein, dt denotes the sampling period of the angular velocity sensor114, and offset is expressed by the following equation.offset=2×position(N)/(N×(N+1)×dt×dt)

Herein, N is the number of samples, and position denotes pos calculatedin the case where offset=0 in the velocity vel and the position pos.

Next, the control unit 140 obtains a maximum amount of change in theposition pos (world coordinate WDC1) in the Z axis direction anddetermines the maximum amount of change as a lower leg length leg_len ofthe user US (step S390).

Herein, in the embodiment, as illustrated in FIGS. 11A and 11B, in aseries of the bending motions, since the ankle JTd attached with thesensor device 100 is allowed to be in a circular motion about the kneeJTc as an axis, the maximum amount of change in the position space maybe determined as the lower leg length leg_len.

In addition, in the embodiment, in the lower limb length estimationprocess, the case where the knee is fixed as the rotation axis and thebending motion is performed so that the lower leg is rotated by 90degrees backward is described. However, the present invention is notlimited thereto.

Namely, as illustrated in FIG. 11C, the lower leg DLr may be bent byabout 180 degrees to the position where the lower leg is in closecontact with the thigh ULr.

In the bending motion of this case, by supporting the lower leg withhands, the lower leg DLr may be lifted up to the 180-degree bentposition. However, in order to increase calculation accuracy of thelower leg length leg_len, it is preferable that the position of the kneeJTc be not moved utmost. A half of the obtained maximum amount of changeof the position pos in the Z axis direction is determined as the lowerleg length leg_len.

In the above-described bending motion, the lower leg may be bent to anangle range of 90 degrees or more and less than 180 degrees withoutbending of the lower leg to 180 degrees. In this case, the maximumamount of change of the position pos (world coordinate WDC1) in the Xaxis direction may be determined as the lower leg length leg_len.

As another method of estimating the lower leg length leg_len, forexample, without attaching the sensor device 100 to the ankle, the userUS carries the sensor device 100 with the hand and moves the sensordevice in the order of ankle position→knee position→ankle position.Similarly to the above-described bending motion, the maximum amount ofchange of the position pos in this motion may be determined as the lowerleg length leg_len.

In this manner, in the embodiment, by executing the above-describedcalibration process, a parameter (attachment orientation correctionmatrix Rc) associated with the sensor device 100 and a parameter(reference posture matrix R5 , lower leg length leg_len) associated withthe body of the user US can be obtained.

The data or information of the parameters obtained by the calibrationprocess is stored in the predetermined storage area of the storage unit150.

Herein, in the case of acquiring none of these parameters, for example,a preset unit matrix (default value) may be applied as the attachmentorientation correction matrix Rc or the reference posture matrix R5.

The lower leg length leg_len may be manually set by the user US, or adefault standard value (default value) may be applied.

Namely, preferably, at least one of the above-described parameters isacquired by the above-described calibration process.

In the case where the reference posture matrix R5 in the one-leg standcannot be acquired, the attachment orientation correction process or thelower leg length estimation process is performed, and the erect posturein two-leg stand may be used as a substitute.

Herein, according to the inventor's test, the calculation accuracy ofthe reference posture matrix R5 is high in the case of performing theone-leg stand. Therefore, theuse of the erect posture in the two-legstand as a substitute is preferred only in the case where the referenceposture matrix R5 in the one-leg stand cannot be obtained.

In addition, in the embodiment, as a method of leading the user toperform a specific calibration motion (bending/stretching motion,one-leg stand, bending motion) in the above-described calibrationprocess, the method of allowing the notification unit 130 to generatethe buzzer sound is described. However, the present invention is notlimited thereto.

Namely, if the user can be led to precisely perform a specific motion bya method, the method of generating sound, vibration, and light fromvarious functional unit (sound unit, vibration unit, light emittingunit, and the like) of the notification unit 130 alone or in combinationthereof to lead the user may be applied.

In the embodiment, the case where the sensor device 100 as a signaldevice executes a series of the process operations associated with theabove-described calibration process is described. However, the presentinvention is not limited thereto.

Namely, the sensor data (acceleration data acc_in and angular velocitydata gyr_in) acquired from the sensor device 100 may be transmitted tothe information processing device 200 through a predeterminedcommunication method, and the information processing device 200 (controlunit 240) may execute a series of the process operations associated withthe above-described calibration process.

(Lower Limb Motion Reproduction Process)

Next, the lower limb motion reproduction process executed in the controlmethod (exercise information reproducing operation) of the exerciseinformation display system described above will be described in detailwith reference to the drawings.

FIG. 12 is a flowchart illustrating an example of a lower limb motionreproduction process applied to the exercise information display systemaccording to one embodiment.

FIG. 13 is a flowchart illustrating an example of a sensor deviceposture estimation process executed in the lower limb motionreproduction process applied to one embodiment.

FIG. 14 is a flowchart illustrating an example of a sensor deviceposition estimation process executed in the lower limb motionreproduction process applied to one embodiment.

FIG. 15 is a flowchart illustrating an example of a lower limb motionestimation process executed in the lower limb motion reproductionprocess applied to one embodiment.

FIGS. 16A to 16E are signal waveform diagrams for explaining acutting-out process for one cycle of running motion in the lower limbmotion reproduction process applied to one embodiment.

FIGS. 17A and 17B are schematic diagrams illustrating a posture of auser during the cutting out of one cycle of running motion in the lowerlimb motion reproduction process applied to one embodiment.

FIG. 18 is a schematic diagram illustrating a position of a referenceposture of one cycle of running motion in the lower limb motionreproduction process applied to one embodiment.

FIG. 19 is a schematic diagram illustrating an example of a table of arelationship of correspondence between a lower leg length and a motionof a base of a thigh referred to in the lower limb motion reproductionprocess applied to one embodiment.

FIG. 20 is a stick model illustrating one cycle of a motion of two lowerlimbs estimated in the lower limb motion reproduction process applied toone embodiment.

The user US sets the sensor device 100 to be in the calibration mode toexecute a series of the calibration processes described above, and afterthat, the user sets the sensor device 100 to be in the measurement modeto do exercise (practice) such as running in the state where the sensordevice 100 is attached.

Alternatively, after the above-described calibration process is ended,the sensor device 100 automatically proceeds to the measurement mode.

Therefore, the sensor data including the acceleration data and theangular velocity data are acquired from the acceleration sensor 112 andthe angular velocity sensor 114 of the sensor unit 110 to be stored inthe predetermined storage area of the storage unit 150 in associationwith time data.

Next, by the user US setting the sensor device 100 to be in thecommunication mode, the attachment orientation correction matrix Rc, thereference posture matrix R5 , the data or information on the parametersof the lower leg length leg_len, and the sensor data acquired during theexercise which are acquired in the calibration process and stored in thestorage unit 150 of the sensor device 100 are transmitted to theinformation processing device 200 through a predetermined communicationmethod.

The parameters or the sensor data are stored in the predeterminedstorage area of the storage unit of the information processing device200 and are used when the information processing device 200 executes thelower limb motion reproduction process.

In the lower limb motion reproduction process applied to the exerciseinformation display system according to the embodiment, as illustratedin the flowchart of FIG. 12, first, the control unit 240 perform aconversion process on the acceleration data acc_in and the angularvelocity data gyr_in transmitted from the sensor device 100 by using theattachment orientation correction matrix Rc for the sensor device 100estimated in the calibration process described above (step S402).

Thus, the corrected acceleration data acc and the corrected angularvelocity data gyr are obtained as expressed by the following equations.acc=Rc×acc_ingyr=Rc×gyr_in

Plural cycles of the signal waveforms of the acceleration data acc andthe angular velocity data gyr in the X, Y, and Z axes obtained throughthe conversion using the attachment orientation correction matrix Rc areillustrated in, for example, FIGS. 16A and 16B.

In the embodiment, in order to prevent error accumulation, thereproduction of the lower limb motion is performed in units of one cycleof the motion (running motion) during the exercise.

Herein, the one cycle of the running motion is a time period from thetime when the leg (left leg) attached with the sensor device 100advances forward through a stroke of the leg motion to the time when theleg advances forward again, that is, a time period of two paces in therunning motion.

By repetitively connecting the one cycle of the motion, a continuousmotion of the lower limb can be reproduced in a pseudo manner.

In the embodiment, one cycle of the running motion (cut-out time periodTc) is cut out from the signal form of the angular velocity data gyrillustrated in FIG. 16B, for example, based on the Y-axis componentillustrated in FIG. 16C (step S404).

More specifically, first, the control unit 240 considers a time periodwhich is sufficiently longer than the one cycle as a cutting-out objectin the signal waveform of the Y-axis component of the angular velocitydata gyr illustrated in FIG. 16C, finds a minimum value (minGyrY≈−10) ofthe angular velocity data gyr, and defines a time position Pa which is ahalf value thereof (GyrY≈−5) as a start point.

Next, the control unit 240 scans from the time position Pa in thedirection of elapse of time (rightward in the figure) and cuts out atime position where the value of the angular velocity data gyr becomes 0to extract as a position Ps.

For example, as illustrated in FIG. 17B, the exercise posture at thestart position Ps is a posture just before the approach to the referenceposture (in FIG. 17A, the posture state where the Z axis of the sensordevice 100 is coincident with the gravitational axis of the worldcoordinate WDC) in the above-described calibration process, and theexercise posture at the start position Ps corresponds to the exerciseposture in the state where the lower limb attached with the sensordevice 100 advances forward.

In the embodiment, the time period between the start position Ps and thenext extracted start position Ps in the direction of elapse of time isextracted as one cycle of the cut-out time period Tc.

Next, the signal waveforms of the one cycle of the cut-out time periodTc are cut out in the acceleration data acc and the angular velocitydata gyr illustrated in FIGS. 16A and 16B.

FIGS. 16D and 16E illustrate examples of the one cycle of the signalwaveforms cut out from the acceleration data acc and the angularvelocity data gyr.

Next, as illustrated in the flowchart of FIG. 12, the control unit 240sequentially executes a process of estimating the posture of the sensordevice 100 based on the angular velocity data gyr (step S406), a processof estimating the position of the sensor device 100 based on theacceleration data acc (step S408), and a process of estimating the lowerlimb motion based on the motion of the lower leg (step S410).

In the posture estimation process for the sensor device 100, asillustrated in FIG. 13, the control unit 240 obtains the posture matrixRp with respect to the world coordinate WDC based on the one cycle ofthe angular velocity data gyr cut out in the cut-out time period Tc.

More specifically, first, the control unit 240 obtains a relativeposture matrix R11 on the basis of the initial start position Ps(exposure posture in the state where the leg advances forward) (stepS422).

Herein, as a method of obtaining the posture matrix on the basis of theinitial posture based on the angular velocity data gyr, the method(refer to step S382) applied in the case of obtaining the posture matrixR7 in the calibration process described above may be applied.

Next, the control unit 240 determines a reference position T1 which is atiming when the lower limb is estimated to have a posture correspondingto the reference posture in the cut-out one cycle (step S424).

Herein, the reference position T1 is a position where the forwardadvancing leg becomes perpendicular to the ground in the Y-Z plane ofthe world coordinate WDC and is set to a position when a time period ofa predetermined ratio (for example, 7% of one cycle) to the cut-out timeperiod Tc elapses from the start point (that is, the start position Ps)of the cut-out one cycle.

In this case, for example, in the signal waveform of the Y-axiscomponent of the angular velocity data gyr illustrated in FIG. 16C, inthe case where the number of frames in one cycle is 143, the 10-th(=143×7%) frame from the start point of the one cycle is set as thereference position T1.

Herein, as the ratio used to define the reference position T1 which is atiming where the posture is estimated to be a posture corresponding tothe reference posture, the ratio of the position elapsed from the startpoint of the one cycle to the cut-out time period Tc is set as follows.

Namely, as illustrated in FIG. 18, if the time of the cut-out one cycle(cut-out time period Tc) is set as 100% and the posture (lower limbmotion) of each frame is tested, even in the case where there areindividual differences in exercise state (running motion) due toexercise skill, physique, age, gender, or the like, it is verified thatthe reference posture is almost included in a range of about 3% to 12%of the one cycle. Therefore, the reference position T1 is preferably setto a position with an arbitrary ratio which is in a range of about 3% to12% of the one cycle.

In the embodiment, it is assumed that, at the reference position T1, theleg of the user US is in the state of the reference posture matrix R5acquired in the calibration process described above.

Herein, in the case where the above-described ratio used to define thereference position T1 is set to 7%, since the 10-th frame is used as areference, the posture of the user US at the reference position T1becomes a coordinate viewed from R5×R11{10} ({ } denotes a function of aframe as a variable).

The control unit 240 obtains the absolute posture of the user US at thereference position T1 by performing coordinate conversion thereof byusing the convert matrix cnvR from the sensor coordinate SNC to theworld coordinate WDC at the time of further erecting to the wirelesscommunication system.

The control unit 240 estimates the posture matrix Rp based on theabsolute posture as expressed by the following equation (step S426).Rp=cnvR×(R5×R11{10})×R11

Herein, the convert matrix cnvR is expressed by the equation (13).

$\begin{matrix}\left\lbrack {{Mathematical}\mspace{14mu}{Formula}\mspace{14mu} 3} \right\rbrack & \; \\{{cnvR} = \begin{bmatrix}0 & 1 & 0 \\0 & 0 & 1 \\1 & 0 & 0\end{bmatrix}} & (13)\end{matrix}$

In addition, in the embodiment, the case where the position when thetime period of the predetermined ratio (for example, 7%) elapses fromthe start point of the cut-out one cycle is set as the referenceposition T1 is described. However, the present invention is not limitedthereto.

For example, as illustrated in FIG. 16D, the position where the X-axiscomponent (that is, the acceleration component in the advancementdirection of the user US) of the acceleration data acc becomes anegative maximum value (minAccX) from the start point of the cut-out onecycle may be set as the reference position T1.

Next, in the position estimation process for the sensor device 100, asillustrated in FIG. 14, the control unit 240 estimates the position ofthe sensor device 100 based on the acceleration data acc.

More specifically, first, the control unit 240 obtains the accelerationdata acc_w by performing coordinate conversion from the sensorcoordinate SNC of the acceleration data acc to the world coordinate WDCby using the posture matrix Rp estimated in the above-described postureestimation process for the sensor device 100 (step S406) as expressed bythe following equation (step S442).acc_w=Rp×acc

Next, since the acceleration data acc_w after the coordinate conversionto the world coordinate WDC include the gravitational accelerationcomponent in the Y axis direction, the control unit 240 obtains theacceleration data acc configured with only the motion componentassociated with the pacing motion by removing the gravitationalacceleration component from the acceleration data acc_w which isconverted to the world coordinate WDC by expressed by the followingequation (step S444).Acc=acc_w−[0 9.8 0]

Next, the control unit 140 converts the acceleration data acc to theposition pos through two-times integration of the acceleration data accof only the motion component (step S446) to obtain the position of thesensor device 100 at each timing.

Namely, the control unit 140 calculates the velocity Vel throughone-time integration of the acceleration data acc configured with onlythe motion component and calculates the position pos through furtherintegration as expressed by the following equations. In the embodiment,a pacing motion such as running having a periodicity is used.Vel=Σ(Acc−aveAcc)×dtPos=Σ(Vel−aveVel)×dt

Herein, aveAcc denotes an acceleration average over one cycle, andaveVel denotes a velocity average.

Next, in the lower limb motion estimation process, as illustrated inFIG. 15, the control unit 240 reproduces in a pseudo manner the lowerlimb motion based on the position pos of the sensor device 100.

Since the sensor device 100 outputting the sensor data during the pacingmotion is attached to the ankle of the user US, the length from theankle to the knee of the user US is already determined based on thelower leg length leg_len calculated in the above-described calibrationprocess (refer to FIG. 9).

The motion of the position of the ankle of the user US is alreadyestimated based on the above-described position estimation process(refer to FIG. 14) of the sensor device 100.

The posture matrix Rp is also already estimated based on theabove-described posture estimation process (refer to FIG. 13) of thesensor device 100.

The orientation from the ankle to the knee of the user US is determinedfrom these data, and the position of the knee is estimated (step S462).

In addition, in general, it is known that the motion of the base of thethigh is much smaller than the motion of the ankle and there arerelatively small individual differences.

Thus, in the embodiment, the relationship between the lower leg lengthand the position of the base of the thigh is predetermined measured, andthe relationship is stored in a table format in the storage unit 250.

More specifically, for example, as illustrated in FIG. 19, a table ofcorrespondence of the lower leg length and the position of the base ofthe thigh over the one cycle is stored as a database in the storage unit250.

Herein, FIG. 19 is a numeric example (unit: mm), for example, in thecase where the one cycle is normalized over 150 frames and the positionof the base of the thigh in each frame is indicated by athree-dimensional coordinate (x, y, z).

By referring to the table, the control unit 240 obtains the position ofthe base of the thigh based on the lower leg length leg_len (step S464).

Thus, the control unit 240 estimates a change in position over the onecycle with respect to three points including the base of the thigh, theknee, and the ankle of the left leg and generates a first reproductionimage where the motion state of the left lower limb is reproduced in apseudo manner.

In addition, with respect to the above-described table, plural types arestored in the storage unit 250 in advance, for example, based on race,gender, age, or the like, and at the time of referring to the table, theuser US may specify an extraction condition by using the inputmanipulation unit 220.

Next, the control unit 240 estimates the change in position of eachpoint of the right leg over the one cycle by delaying (shifting) theposition of each point of the left leg by a half period in the timedirection and estimates the motion state of the right lower limb togenerate a second reproduction image which is to be reproduced in apseudo manner (step S466).

Next, the control unit 240 combines the first reproduction image and thesecond reproduction image and connects the bases of the thighs of thetwo legs with a desired distance (step S468) and estimates the motionstate of the two lower limbs over the one cycle as illustrated in FIG.20 to generate a reproduction image which is to be reproduced in apseudo manner.

Herein, in FIG. 20, for simplifying the illustration, a stick model (orskeleton model) is used, the left leg attached with the sensor device100 is indicated by a solid line, and the right leg is indicated by adotted line. The data of the estimated one cycle of the motion state ofthe two lower limbs are stored in the predetermined storage area of thestorage unit 250.

(Example of Display of Exercise Information)

Next, an example of display of the exercise information applied to theabove-described control method for the exercise information displaysystem will be described with reference to the drawings.

FIG. 21 is a schematic diagram illustrating an example of exerciseinformation displayed in the information processing device applied tothe exercise information display system according to the embodiment.

The motion state of the two lower limbs estimated in the above-describedlower limb motion reproduction process is displayed in the display unit210 of the information processing device 200 in a predetermined displayformat.

More specifically, for example, as illustrated in FIG. 21, the controlunit 240 reads data on the currently estimated one cycle of the motionstate of the two lower limbs from the storage unit 250 and displays thedata in a first display area 211 in a format of an animation(hereinafter, referred to as “stick animation”) of the reproductionimages using a stick model.

The control unit 240 reads data on the motion state of the lower limbduring the previous exercise of the user US as a comparison object inadvance in the storage unit 250 and displays the data in a seconddisplay area 212 in a format of a stick animation.

In addition, in the storage unit 250, the one cycle of data are isstored so that the posture of the start position during the exercise ofthe comparative motion state of the lower limb is coincident with thatof the currently estimated motion state of the lower limb.

Thus, for example, as illustrated in FIG. 21, the control unit 240arranges the first display area 211 and the second display area 212 tobe adjacent to the screen of the display unit 210 in the horizontaldirection (left-right direction in the figure) and performs the displayso that the currently estimated motion state of the lower limb and thecomparative motion state of the lower limb can be compared.

In addition, the control unit 240 synchronizes the one cycle of the readdata with respect to the currently estimated motion state of the lowerlimb and the one cycle of the read data the comparative motion state ofthe lower limb and continuously repeats, so that the motion state of thelower limb of the user US and the comparative motion state of the lowerlimb are displayed according to the stick animation.

In order to facilitate recognition or determination of the motion stateof the lower limb, joints such as the base JTb of the thigh and the kneeJTc or the like are displayed in the stick animation.

Therefore, the user US can compare and view the currently estimatedmotion state of the lower limb and the comparative motion state of thelower limb displayed in the display unit 210, and thus, can preciselydetermine defects, problems, or the like of the motion state (runningmotion) of the lower limb during the exercise so that the user canreflect the defects, the problems or the like on the subsequent trainingor the like.

Herein, in FIG. 21, the currently estimated motion state of the lowerlimb and the comparative motion state of the lower limb are displayed inthe stick animations drawn by the same type of lines. However, thepresent invention is not limited thereto. In order to facilitate therecognition or determination of the motion state of the lower leg,various display forms may be applied or added.

For example, the lower limbs of the left and right legs may be displayedwith different colors or difference types of lines, or the motion of theupper body in association with the motion state of the lower limb (lowerbody) may be added in display.

In addition, the data which the control unit 240 reads from the storageunit 250 as a comparison object and displays in the second display area212 in a format of the stick animation are not limited to the dataduring the previous exercise of the user US. For example, data on themotion state of the lower limb of an athlete (elite runner) havinghigher pacing performance than the user US may be used. In this case,the motion state of the lower limb during the exercise of the user UScan be compared with the motion state of the lower limb of the eliterunner, and thus, the defects, the problems, or the like of the motionstate can be more precisely determined, so that the defects, theproblems, or the like can be reflected well on the subsequent trainingor the like.

In the embodiment, the case where the stick animation where thecurrently estimated motion state of the lower limb is reproduced in apseudo manner and the stick animation where the comparative motion stateof the lower limb is reproduced in a pseudo manner are arranged in thedisplay unit 210 in the horizontal direction is illustrated. However,the present invention is not limited thereto, but the two stickanimations may be displayed to superpose (overlap) each other.

As the information displayed in the display unit 210, in addition to thestick animation of the currently estimated motion state of the lowerlimb and the stick animation of the comparative motion state of thelower limb, various exercise information based on the sensor dataacquired in the measurement mode, for example, numerical values of amaximum kick-up height of the leg, a stride, a pitch (the number ofsteps per unit time), or the like may be displayed.

In the case where the above-described lower limb motion reproductionprocess is executed by using a cloud system, the information processingdevice 200 may be allowed to activate a web browser as software forbrowsing network information to display exercise information(reproduction image of the lower limb motion or the like) generated asweb display data in the cloud system on a web screen of the display unit210.

In this manner, in the embodiment, by the simple, low-cost configurationwhere one sensor device including at least the acceleration sensor andthe angular velocity sensor is attached to the ankle of the user US, themotion state during the exercise can be precisely determined and can beaccurately reproduced, so that the user can reflect the motion state onimprovement or the like of training based on the reproduction image inthe exercise state.

In particular, in the embodiment, even in the case where the sensordevice attached to the ankle of the user is shifted from the normalattachment orientation, without changing the attachment orientation thesensor data acquired during the exercise is corrected by using theattachment orientation correction matrix for the sensor device estimatedin the calibration process, so that trouble to correct the attachmentorientation of the sensor device is omitted, and the position locus ofthe sensor device is estimated at a good accuracy, so that the motion ofthe lower limb of the user can be accurately reproduced.

In the embodiment, even in the case where the lower leg length varieswith persons, the lower leg length of the user is estimated in thecalibration process, so that the motion of the lower limb of the usercan be reproduced with a stick animation based on the position locus ofthe sensor device at a good accuracy.

In the embodiment, the reference posture of a series of the motionduring the exercise of the user is set in the calibration process, andcorrection of removing the gravitational acceleration component includedin the sensor data acquired during the exercise is performed, and thus,the position locus of the sensor device is estimated at a good accuracy,so that the motion of the lower limb of the user can be accuratelyreproduced.

<Modified Example of Exercise Information Display System>

Next, a modified example of the exercise information display systemaccording to the embodiment will be described.

In the above-described embodiment, the case the sensor device 100 is setto the predetermined operation mode by the user US manipulating theinput manipulation unit 120 (manipulation buttons 122 and 124) of thesensor device 100 attached to the ankle is described.

In the modified example of the embodiment, in addition to the sensordevice 100, a manipulation device attached to different sites of thebody or carried is included, and operation mode setting manipulation forthe sensor device 100 or recognition of the operation state thereof isperformed through the manipulation device.

FIGS. 22A, 22B, and 22C are schematic configurational diagramsillustrating a modified example of the exercise information displaysystem according to the embodiment. FIG. 22A is a schematic diagramillustrating an example where a sensor device and a manipulation deviceapplied to the modified example are attached to a body, FIG. 22B is aschematic diagram illustrating an outer appearance of the manipulationdevice, and FIG. 22C is a schematic block diagram illustrating aconfigurational example of the manipulation device.

Herein, the same components as those of the above-described embodiments(refer to FIGS. 1A, 1B, and 1C and FIGS. 2A and 2B) are denoted by thesame or equivalent reference numerals, and the description thereof issimplified.

For example, as illustrated in FIGS. 22A and 22B, an exerciseinformation display system according to the modified example isconfigured to include a sensor device 100 attached to an ankle of a userUS and a manipulation device 300 attached to another site of the body ofthe user US or carried by the user US to manipulate the sensor device100.

In addition, although not shown, similarly to the above-describedembodiment (refer to FIGS. 1A, 1B, and 1C), the exercise informationdisplay system is configured to include an information processing device200 which reproduces in a pseudo manner an exercise state based onsensor data or the like acquired from the sensor device 100 and providesthe exercise state to the user.

As illustrated in FIGS. 22A and 22B, the manipulation device 300 is anelectronic device, for example, having a wristwatch-type orwristband-type outer appearance attached to a wrist or the like of theuser US.

In addition, in the modified example, as an example of the manipulationdevice 300, a wristwatch-type or wristband-type electronic device isillustrated. However, the present invention is not limited thereto, butthe manipulation device 300 may be, for example, electronic deviceshaving different shapes attached to various sites of the body such as anecklace type attached to the neck, a sport glasses type having aneyeglasses shape, an earphone type attached to the ears, or a mobiledevice such as a smartphone may be applied.

More specifically, for example, as illustrated in FIG. 22C, themanipulation device 300 is configured mainly to include a display unit310, an input manipulation unit 320, a notification unit 330, a controlunit 340, a storage unit 350, a communication OF unit 360, and a powersupply unit 370.

Herein, the manipulation device 300 has at least the functions of theinput manipulation unit 120 and the notification unit 130 among thefunctions of the sensor device 100 illustrated in the above-describedembodiment (refer to FIGS. 2A and 2B).

Therefore, the sensor device 100 applied to this modified example mayhave a configuration where the input manipulation unit 120 and thenotification unit 130 are not included.

Namely, the display unit 310 includes a display panel, and thenotification unit 330 includes a sound unit, a vibration unit, a lightemitting unit, or the like.

The display unit 310 and the notification unit 330 provides or notifies,to the user, at least the information on the procedure in theabove-described calibration process, information on operationabnormality of the sensor device 100, or the like through visualperception, auditory perception, tactile perception, and the like. Thedisplay unit 310 and the notification unit 330 may provide or notify, tothe user, the information or the like on various operations (thecalibration process for a predetermined parameter, the sensing operationof the sensor unit 110, transmission operation for the measured sensordate to the information processing device 200, or the like) of thesensor device 100 of which operation mode is set by manipulating theinput manipulation unit 320 through visual perception, auditoryperception, tactile perception, and the like. Herein, the informationprovided or notified by the notification unit 330 may be interlockedwith the display of the display unit 310.

For example, as illustrated in FIG. 22B, input manipulation unit 320includes a push button or a slide button installed on a side surface ora front surface of a case of the manipulation device 300, a touch-paneltype switch installed on the front surface side (viewing side) of thedisplay unit 310, or the like.

The input manipulation unit 320 is used for manipulation for setting atleast an operation mode of the sensor device 100.

The input manipulation unit 320 is also used for manipulation or thelike for setting items which are displayed in the display unit 310.

The control unit 140 transmits a control signal to the sensor device 100through a communication I/F unit 360 described later according to themanipulation of the input manipulation unit 320 and performs controllingof setting the operation mode of the sensor device 100.

The control unit performs controlling of allowing the display unit 310and the notification unit 330 to provide or notify, to the user,information on various operations such as a calibration process for apredetermined parameter executed in the sensor device 100, a sensingoperation of the sensor device 100, transmission operation for thesensor data or the like to the information processing device 200.

The storage unit 350 stores a program for executing the above-describedoperations or various data or information used or generated when thecontrol unit 140 executes the program.

The communication I/F unit 360 functions as an interface at the time oftransmitting the control signal for setting the operation mode of thesensor device 100 or at the time of receiving information on a procedureof the calibration process executed in the sensor device 100.

Herein, as a method of transmitting/receiving signals between themanipulation device 300 and the sensor device 100 through thecommunication I/F unit 360, for example, various wireless communicationmethods such as Bluetooth (registered trademark) or Bluetooth low energy(LE) may be applied.

The power supply unit 370 supplies driving power to each component ofthe manipulation device 300. Similarly to the above-described powersupply unit 370 of the sensor device 100, as the power supply unit 370,well-known primary battery or secondary battery, power sources accordingto energy harvesting technique, or the like may be applied.

In the exercise information display system having the aboveconfiguration, besides the functions and effects of the above-describedembodiment, the following characteristic functions and effects can beobtained.

Namely, in the modified example, the user US can set the operation modeof the sensor device 100 by manipulating the hand-held manipulationdevice 300 attached to the wrist or carried with the hand in the statewhere the sensor device 100 is attached to the ankle.

In the modified example, when the calibration process for apredetermined parameter in the sensor device 100 is executed, theinformation on the procedure can be provided or notified to the user USthrough the hand-held manipulation device 300.

Namely, according to the modified example, without bending the body ofthe user US to directly manipulate the sensor device 100 attached to theankle, the user US can simply and securely perform a specific motion(erecting motion, bending/stretching motion, or the like) associatedwith the setting of the operation mode of the sensor device 100 or thecalibration process, so that load of the user US can be reduced at thetime of using the exercise information display system.

In addition, in the above-described embodiment, as the exercisereproduced in a pseudo manner by the exercise information displaysystem, running is exemplified in the description.

However, the present invention is not limited thereto, but the presentinvention may be applied to various exercises such as walking, cycling,or swimming

Heretofore, while some embodiments of the present invention aredescribed, the present invention is not limited to the embodimentdescribed above, but the invention disclosed in the claims and theequivalence thereof is included.

What is claimed is:
 1. An exercise information display systemcomprising: a sensor unit which is attached to an ankle of one leg of ahuman body and outputs data on a motion state of the one leg; a displayunit; and a control unit which processes the data, wherein the controlunit performs: acquiring the data as calibration data when the humanbody performs a calibration motion for acquiring a parameter expressingat least one of an attachment orientation of the sensor unit, anattachment position of the sensor unit on the one leg, and a posture ofthe one leg when standing on the one leg, acquiring the parameter basedon the calibration data, acquiring the data as exercise data when thehuman body performs an exercise of moving at least the one leg,generating a first animation where the motion state of the one legduring the exercise is reproduced in a pseudo manner based on theexercise data and the parameter, estimating a motion state of the otherleg during the exercise based on the first animation, and generating asecond animation where the motion state of the other leg which isestimated is reproduced in a pseudo manner, generating a reproductionanimation where a motion state of the two legs during the exercise isreproduced in a pseudo manner by combining the first animation and thesecond animation, and displaying at least the reproduction animation asexercise information on the display unit.
 2. The exercise informationdisplay system according to claim 1, wherein: the calibration motionincludes a first motion where the two legs are bent/stretched in a statewhere two knees of the two legs are kept together, and the control unitacquires a correction matrix for correcting a difference of theattachment orientation of the sensor unit from a correct attachmentorientation as the parameter based on first data output from the sensorunit when the human body performs the first motion as the calibrationmotion.
 3. The exercise information display system according to claim 2,wherein the control unit acquires corrected first data of one cycle ofthe motion of the human body by converting the first data output fromthe sensor unit during the exercise by using the correction matrix andgenerates the reproduction animation based on the corrected first data.4. The exercise information display system according to claim 1,wherein: the calibration motion includes a second motion where the oneleg is in an erect posture by allowing a sole of a foot of the one legto be in contact with a ground and the other leg is separated from theground, and the control unit acquires a posture matrix expressing thereference posture as the parameter based on second data output from thesensor unit when the human body performs the second motion as thecalibration motion.
 5. The exercise information display system accordingto claim 4, wherein the control unit estimates a posture of the sensorunit during the exercise in an absolute coordinate by using the posturematrix and generates the reproduction animation based on the estimatedposture of the sensor unit.
 6. The exercise information display systemaccording to claim 1, wherein: the calibration motion includes a thirdmotion where the one leg is in an erect posture by allowing a sole of afoot of the one leg to be in contact with a ground, after that, rotatinga lower leg of the one leg at least 90 degrees by using a knee of theone leg as a rotation axis, and after that, returning the one leg to anerect posture by allowing the sole of the foot of the one leg to be incontact with the ground, and the control unit acquires a length betweenthe knee of the one leg and the attachment position of the sensor unitas a lower leg length of the one leg as the parameter based on thirddata output from the sensor unit according to the motion of the humanbody when the human body performs the third motion as the calibrationmotion.
 7. The exercise information display system according to claim 6,wherein the control unit generates the reproduction animation based onthe acquired lower leg length.
 8. The exercise information displaysystem according to claim 1, wherein: the calibration motion includes: afirst motion where the two legs are bent/stretched in a state where twoknees of the two legs are kept together; a second motion where the oneleg is in an erect posture by allowing a sole of a foot of the one legto be in contact with a ground and the other leg is separated from theground; and a third motion where the one leg is in an erect posture byallowing the sole of the foot of the one leg to be in contact with theground, after that, rotating a lower leg of the one leg at least 90degrees by using the knee of the one leg as a rotation axis, and afterthat, returning the one leg to an erect posture by allowing the sole ofthe foot of the one leg to be in contact with the ground, and thecontrol unit: acquires a correction matrix for correcting a differenceof the attachment orientation of the sensor unit from a correctattachment orientation as the parameter based on first data output fromthe sensor unit according to the motion of the human body when the humanbody performs the first motion as the calibration motion, acquires aposture matrix defining the reference posture based on second dataoutput from the sensor unit according to the motion of the human bodywhen the human body performs the second motion as the calibrationmotion, acquires a length between the knee of the one leg and theattachment position of the sensor unit as a lower leg length of the oneleg as the parameter based on third data output from the sensor unitaccording to the motion of the human body when the human body performsthe third motion as the calibration motion, acquires corrected firstdata and corrected second data of one cycle of the motion of the humanbody by converting the first data and the second data by using thecorrection matrix, estimates a posture of the sensor unit during theexercise in an absolute coordinate by using the posture matrix,estimates a position of the sensor unit during the exercise in theabsolute coordinate by removing a gravitational acceleration componentfrom the corrected second data and performing two-times integration, andgenerates the reproduction animation of the one cycle based on theestimated posture and position of the sensor unit, the acquired lowerleg length, and a known relationship between a lower leg length and aposition of a base of a thigh of the human body.
 9. An exerciseinformation display method comprising: acquiring data output from asensor unit as calibration data attached to an ankle of one leg of ahuman body when the human body performs a calibration motion foracquiring a parameter expressing at least one of an attachmentorientation of the sensor unit, an attachment position of the sensorunit on the one leg, and a posture of the one leg when standing on theone leg; acquiring the parameter based on the calibration data;acquiring the data as exercise data when the human body performs anexercise of moving at least the one leg; generating a first animationwhere the motion state of the one leg during the exercise is reproducedin a pseudo manner based on the exercise data and the parameter;estimating a motion state of the other leg during the exercise based onthe first animation, and generating a second animation where the motionstate of the other leg which is estimated is reproduced in a pseudomanner; generating a reproduction animation where a motion state of thetwo legs during the exercise is reproduced in a pseudo manner bycombining the first animation and the second animation; and displayingat least the reproduction animation as exercise information on a displayunit.
 10. The exercise information display method according to claim 9,wherein: the calibration motion includes a first motion where the twolegs are bent/stretched in a state where two knees of the two legs arekept together, and the acquiring the parameter includes acquiring acorrection matrix for correcting a difference of the attachmentorientation of the sensor unit from a correct attachment orientation asthe parameter based on first data output from the sensor unit when thehuman body performs the first motion as the calibration motion.
 11. Theexercise information display method according to claim 10, wherein thegenerating the reproduction animation includes acquiring corrected firstdata of one cycle of the motion of the human body by converting thefirst data output from the sensor unit during the exercise by using thecorrection matrix and generating the reproduction animation based on thecorrected first data.
 12. The exercise information display methodaccording to claim 9, wherein: the calibration motion includes a secondmotion where the one leg is in an erect posture by allowing a sole of afoot of the one leg to be in contact with a ground and the other leg isseparated from the ground, and the acquiring the parameter includesacquiring a posture matrix expressing the reference posture as theparameter based on second data output from the sensor unit when thehuman body performs the second motion as the calibration motion.
 13. Theexercise information display method according to claim 12, wherein thegenerating the reproduction animation includes estimating a posture ofthe sensor unit during the exercise in an absolute coordinate by usingthe posture matrix and generating the reproduction animation based onthe estimated posture of the sensor unit.
 14. The exercise informationdisplay method according to claim 9, wherein: the calibration motionincludes a third motion where the one leg is in an erect posture byallowing a sole of a foot of the one leg to be in contact with a ground,after that, rotating a lower leg of the one leg at least 90 degrees byusing a knee of the one leg as a rotation axis, and after that,returning the one leg to an erect posture by allowing the sole of thefoot of the one leg to be in contact with the ground, and the acquiringthe parameter includes acquiring a length between the knee of the oneleg and the attachment position of the sensor unit as a lower leg lengthof the one leg as the parameter based on third data output from thesensor unit according to the motion of the human body when the humanbody performs the third motion as the calibration motion.
 15. Theexercise information display method according to claim 14, wherein thereproduction animation is generated based on the acquired lower leglength.
 16. The exercise information display method according to claim9, wherein: the calibration motion includes: a first motion where thetwo legs are bent/stretched in a state where two knees of the two legsare kept together; a second motion where the one leg is in an erectreference posture by allowing a sole of a foot of the one leg to be incontact with a ground and the other leg is separated from the ground;and a third motion where the one leg is in an erect posture by allowingthe sole of the foot of the one leg to be in contact with the ground,after that, rotating a lower leg of the one leg at least 90 degrees byusing the knee of the one leg as a rotation axis, and after that,returning the one leg to an erect posture by allowing the sole of thefoot of the one leg to be in contact with the ground, the acquiring theparameter includes: acquiring a correction matrix for correcting adifference of the attachment orientation of the sensor unit from acorrect attachment orientation as the parameter based on first dataoutput from the sensor unit according to the motion of the human bodywhen the human body performs the first motion as the calibration motion;acquiring a posture matrix defining the reference posture based onsecond data output from the sensor unit according to the motion of thehuman body when the human body performs the second motion as thecalibration motion; and acquiring a length between the knee of the oneleg and the attachment position of the sensor unit as a lower leg lengthof the one leg as the parameter based on third data output from thesensor unit according to the motion of the human body when the humanbody performs the third motion as the calibration motion, and thegenerating the reproduction animation includes: acquiring correctedfirst data and corrected second data of one cycle of the motion of thehuman body by converting the first data and the second data by using thecorrection matrix; estimating a posture of the sensor unit during theexercise in an absolute coordinate by using the posture matrix;estimating a position of the sensor unit during the exercise in theabsolute coordinate by removing a gravitational acceleration componentfrom the corrected second data and performing two-times integration; andgenerating the reproduction animation of the one cycle based on theestimated posture and position of the sensor unit, the acquired lowerleg length, and a known relationship between a lower leg length and aposition of a base of a thigh of the human body.
 17. A non-transitorycomputer-readable recording medium storing an exercise informationdisplay program for causing a computer to execute: acquiring data from asensor unit as calibration data, the sensor unit being attached to anankle of one leg of a human body, when the human body performs acalibration motion for acquiring a parameter expressing at least one ofan attachment orientation of the sensor unit, an attachment position ofthe sensor unit on the one leg, and a posture of the one leg whenstanding one the one leg; acquiring the parameter based on thecalibration data; acquiring the data as exercise data when the humanbody performs an exercise of moving at least the one leg; generating afirst animation where the motion state of the one leg during theexercise is reproduced in a pseudo manner based on the exercise data andthe parameter; estimating a motion state of the other leg during theexercise based on the first animation, and generating a second animationwhere the motion state of the other leg which is estimated is reproducedin a pseudo manner; generating a reproduction animation where a motionstate of the two legs during the exercise is reproduced in a pseudomanner by combining the first animation and the second animation; anddisplaying at least the reproduction animation as exercise informationon a display unit.